UNITED STATES PATENT APPLICATION 

for 



AN APPARATUS FOR OPTICAL NAVIGATION 



Inventors: 

TONG XIE 
MICHAEL BROSNAN 



10030187-1 



AN APPARATUS FOR OPTICAL NAVIGATION 
FIELD OF INVENTION 

5 Various embodiments in accordance with the invention relate to the 

field of optical navigation. 

BACKGROUND OF THE INVENTION 

10 As computing technology continues to advance, computing devices 

with new form factors are introduced into the marketplace. These new form 
factors often provide users with new ways of computer-human interaction. 
For example, a large number of computing devices introduced in recent 
years have display screens that are also used for data entry. For example, 

15 personal digital assistants (PDAs) and tablet personal computers (PCs) 
typically have displays screens that function both as displays and as 
digitizers for receiving data. 

Many existing screen input and navigation technologies are based on 
20 touch screen technology. A touch screen is a computer display screen that 
is sensitive to touch, for example, touching a stylus to the touch screen. In 
order to provide screen input and navigation functionality, touch screens are 
specially constructed screens. Current touch screen technology includes 
resistive, capacitive, and surface acoustic sensing touch screen panels. 
25 Specifically, touch screens may require special membranes, transparent 
conductive films, and large source and sensor arrays in addition to 
standard display manufacturing. 

There are several drawbacks to requiring a touch screen for providing 
30 data input and navigation of a PDA or a tablet PC. Touch screens are 
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typically expensive to manufacture, due to the sensitivity of the components. 
The cost of a touch screen increases dramatically as the size increases. 
Furthermore, current touch screen technology is typically only applied to 
build special displays and cannot readily retrofit existing monitors. Touch 
5 screens are also very sensitive to contamination during operation, which 
can lead to in costly repairs and to computer system downtime. 
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SUMMARY OF THE INVENTION 

Various embodiments in accordance with the invention, an apparatus 
for optica! navigation and data input on an illuminated surface, are 
5 described. By providing for optical navigation and data input, computer 
systems using display screens for navigation and data entry do not require 
the use of touch screen technology. Furthermore, by utilizing the 
illumination of a self-illuminated surface, it is not necessary to provide 
illumination from an internal light source within the electronic device. By not 
10 requiring an internal light source, electronic devices for optical navigation 
and data entry in embodiments in accordance with the invention are 
provided with significant power savings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorporated in and form a 
part of this specification, illustrate embodiments in accordance with the 
5 invention and, together with the description, serve to explain the principles of 
the invention: 

FIG. 1A illustrates an exemplary system for optical navigation on an 
illuminated surface of an embodiment in accordance with the invention. 

10 

FIG. 1B illustrates an exemplary electronic device for optical 
navigation on an illuminated surface of an embodiment in accordance with 
the invention. 

15 FIG. 2A is a block diagram of optical navigation componentry in an 

embodiment in accordance with the invention. 

FIG. 2B is a block diagram of optical navigation componentry 
comprising an internal infrared (IR) illumination source in an embodiment 
20 in accordance with the invention 

FIG. 3 is a diagram of an optical motion detection circuit of an 
embodiment in accordance with the invention. 

25 FIG. 4A is a diagram of an exemplary cathode ray tube (CRT) upon 

which an embodiment in accordance with the invention may be 
implemented. 

FIG. 4B is a diagram of an exemplary shadow mask of a CRT upon 
30 which an embodiment in accordance with the invention may be 
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implemented. 

FIG. 4C is a diagram of an exemplary liquid crystal display (LCD) 
upon which an embodiment in accordance with the invention may be 
5 implemented. 

FIG. 5A and FIG. 5B are flow charts illustrating a process of optical 
navigation on an illuminated surface using an electronic device of an 
embodiment in accordance with the invention. 

10 
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DETAILED DESCRIPTION OF THE INVENTION 

Various embodiments in accordance with the invention provide an 
electronic device for optical navigation and data entry on an illuminated 
5 surface. Accordingly, computer systems using display screens for 

navigation and data entry do not require the use of touch screen technology, 
which can be very expensive and require special manufacturing processes. 
Furthermore, by utilizing the illumination of a self-illuminated surface, it is 
not necessary to provide illumination from an internal light source of the 
10 electronic device. Illumination sources are typically a substantial power 
drain on wireless optical navigation devices. By not requiring an internal 
illumination source, electronic devices for optical navigation and data entry 
in embodiments in accordance with the invention are provided with 
significant power and cost savings. 

15 

For purposes of clarity in describing embodiments in accordance 
with the invention, the application starts with a discussion of the physical 
structure of an embodiment of an optical screen navigation device in 
accordance with the invention. This is followed by a description of the 
20 physical structure of exemplary illuminated surfaces upon which 

embodiments in accordance with the invention may be implemented. That 
discussion will then be followed by a description of the operation of the 
optical screen navigation device on an illuminated surface in an 
embodiment in accordance with the invention. 

25 

Physical Structure of Optical Screen Navigation Device 
in Embodiments in Accordance with the Invention 

30 FIG. 1A illustrates an exemplary system 100 for optical navigation on 
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an illuminated surface in one embodiment in accordance with the invention. 
System 100 comprises housing 110, display screen 120 and optical screen 
navigation device 130. It should be appreciated that embodiments in 
accordance with the invention are directed towards a using optical screen 
5 navigation device 130 in conjunction with display screen 120 to navigate 
and input data on a computer system. While optical screen navigation 
device 130 is shown having a form factor similar to a pen, it should be 
appreciated that optical screen navigation device 130 is not limited to a pen 
implementation, and may be implemented as having any form factor, such 
10 as a mouse. 

In one embodiment in accordance with the invention, display screen 
120 is an integral display screen of a computing device, such as a tablet 
personal computer (PC) or a personal digital assistant. In another 
15 embodiment in accordance with the invention, display screen 120 is an 
external display screen of communicatively coupled to a computing device, 
such as a desktop PC. Display screen 120 may be a liquid crystal display 
(LCD), a cathode ray tube (CRT), or any other illuminated surface for 
displaying data of a computer system. 

20 

While embodiments in accordance with the invention are directed 
towards using optical screen navigation device 130 in conjunction with 
display screen 120 to navigate and input data on a computer system, it 
should be understood that display screen 120 may be any illuminated 
25 surface. Optical screen navigation device 130 in conjunction with an 

illuminated display can be used to navigate and input data on a different 
display screen. 
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In one embodiment in accordance with the invention, optical screen 
navigation device 130 is coupled to display housing 110 by cable 140. 
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Cable 140 may provide data transfer between optical screen navigation 
device 130 and a computer system (e.g., a tablet PC) residing within display 
housing 110. It should be appreciated that the shown embodiment is 
exemplary, and that optical screen navigation device 130 may be connection 
5 to a computer system by a wired (e.g., cable 140) or a wireless connection. 

FIG. 1B illustrates an exemplary optical screen navigation device 130 
for optical navigation on an illuminated surface in an embodiment in 
accordance with the invention. Optical screen navigation device comprises 
10 optical navigation componentry 165 for navigation and data input. In one 
embodiment in accordance with the invention, optical screen navigation 
device 130 comprises aperture 185 on surface 180 for providing optical 
navigation componentry 165 access to an illuminated surface. 

15 In one embodiment in accordance with the invention, optical screen 

navigation device 130 is connected to a computer system over a wired 
connection using cable 140. Cable 140 may provide data communication 
and/or power to optical screen navigation device 130. Power is provided to 
optical navigation componentry 165 over power bus 155 and 

20 communication to and from optical navigation componentry 165 is 
supported over communication bus 160. 

In another embodiment in accordance with the invention, optical 
screen navigation device 130 may comprise internal power source 150 and 

25 wireless transceiver 170 for communicating with a computer system. 

Power is provided to optical navigation componentry 165 over power bus 
155 and communication to and from optical navigation componentry 165 is 
supported over communication bus 160. It should be appreciated that 
optical screen navigation device 130 may comprise any combination of 

30 internal or external power sources and wired or wireless data 
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communication. 

FIG. 2A is a block diagram of optical navigation componentry 165a in 
an embodiment in accordance with the invention. As described above, 
5 surface 180 comprises aperture 185 for providing optical access to self- 
illuminated surface 206. Although it has been omitted for clarity, the 
aperture 185 might include a window that is transparent for receiving light 
from self-illuminated surface 206, and which would serve to keep dust, dirt 
or other contamination out of the housing cavity of optical screen navigation 
10 device 130. 

Optical navigation componentry 165a comprises optical motion 
detection circuit 210 and optical element 208. FIG. 3 is a diagram of an 
optical motion detection circuit 210 of an embodiment in accordance with 

15 the invention. Optical motion detection circuit 210 comprises detector 304 
and image processor 306. Detector 304 is operable to capture an image of 
a surface (e.g., self-illuminated surface 206 of Figures 2A and 2B). It should 
be appreciated that detector 304 may be a charged coupled device, an 
amorphous photodiode array, or any other type of array detectors known in 

20 the art. Detector 304 captures images (e.g., image 302) at a specified rate, 
for example, two thousand images per second. Each image is forwarded to 
image processor 306, which is operable to determine positional 
information 310 based on the relative motion of each image. Positional 
information 310 may be transmitted to a computer system over a wired 

25 connection (e.g., cable 140 of Figures 1A and 1B) or a wireless connection 
(e.g., wireless transceiver 170 of Figure 1B). Optionally, optical motion 
detection circuit 210 may comprise optical filter 308. 

With reference to FIG. 2A, self-illuminated surface 206 is any surface 
30 that has texture and structures sufficient for performing motion sensing. 
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Generally, any micro textured surface having features whose size falls within 
the range of 5 to 500 microns is adequate for use with optical screen 
navigation device 130. It should be appreciated that useful, high-contrast 
images of surface texture can be generated by detecting structural 
variations that are inherent to the surface or are formed on the surface. For 
example, images may be formed based upon the contrast between 
shadows in valleys and bright spots at the peaks of inherent structural 
features. Such features are typically microscopic in nature, often ranging 
betweenIO urn and 40 in size on common printed media. As an 
alternative, speckle may be used, since specular reflection of a coherent 
beam produces a contrast pattern of bright and dark areas. A third source 
of contrast information is color. Color contrast is independent of surface 
texture. Even when illuminating the texture-free surface with light in the 
visible range, color contrast exists between regions of different colors, e.g., 
between different shades of gray. 

Self-illuminated surface 206 receives illumination from illumination 
source 212. In one embodiment, self-illuminated surface 206 is a surface 
of a display screen. Since display screens can differ substantially in design 
and manufacture, there are a wide variety of surfaces that are satisfactory for 
use as self-illuminated surface 206. Some of the useful surfaces are 
described below at Figures 4A through 4C. 

FIG. 2B is a block diagram of optical navigation componentry 165b 
comprising a supplemental light source 222 in an embodiment in 
accordance with the invention. In one embodiment, optical navigation 
componentry comprises optical element 224 for focusing interference 
reducing illumination from interference reduction light source 22 over an 
area of self-illuminated surface 206. Optical navigation componentry 165b 
comprises the same components as optical navigation componentry 165a 
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of FIG. 2A, in addition to supplemental light source 222. Optionally, optical 
navigation componentry 165b comprises optical filter 230. 

In one embodiment in accordance with the invention, optical motion 
5 detection circuit 210 is operable to detect whether illumination at self- 
illuminated surface 206 is sufficient to capture an image. In some 
instances, the illumination of self-illuminated surface 206 may not be 
sufficient to allow optical screen navigation device 130 to capture an image. 
For example, the illumination may be to dim. The photo detectors can 
10 detect this condition and control additional components of optical navigation 
componentry 165b to compensate for the insufficient illumination. To 
account for insufficient illumination at self-illuminated surface 206, optical 
navigation componentry 165b utilizes supplemental light source 222 to 
provide additional illumination onto self-illuminated surface 206. 

15 

In another embodiment in accordance with the invention, optical 
motion detection circuit 210 is operable to detect interference at self- 
illuminated surface 206 caused by illumination source 212. In some 
instances, the illumination of self-illuminated surface 206 can interfere with 
20 the performance of optical screen navigation device 130. For example, the 
illumination may modulate at a frequency that disrupts the performance of 
optical motion detection circuit 210. The photo detectors can detect this 
disruption and control additional components of optical navigation 
componentry 165b to compensate for this interference. 

25 

To account for interference from illumination source 212, optical 
navigation componentry 165b utilizes supplemental light source 222 as an 
interference reduction light source in conjunction with an optical filter. In 
one embodiment, the optical filter is an optical filter element located within 
30 optical navigation componentry 165b (e.g., optical filter 230). In another 
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embodiment, the optical filter is an optical filter element located within 
optical motion detection circuit 210 (e.g., optical filter 308). In another 
embodiment, the filtering functionality is implemented electronically within 
image processor 306 of Figure 3. It should be appreciated that 
supplemental light source 222 can operate as an interference reduction 
light source while providing supplemental illumination where illumination 
on self-illuminated surface 206 is insufficient, as described above. 

As described above, the illumination of self-illuminated surface 206 
may have a detrimental effect on the performance of optical motion 
detection circuit 210. To reduce or eliminate interference caused by the 
illumination of self-illuminated surface 206, the optical filter can be used 
filter out light received at particular wavelengths or frequencies. In one 
embodiment, supplemental light source 222 is an infrared light source that 
emits light at a known frequency. The optical filter is operable to filter out the 
illumination received at the known frequency, and blocking the disruptive 
illumination from illumination source 212. By blocking out the interfering 
illumination and receiving only the infrared illumination, interference with 
optical motion detection circuit 210 can be eliminated. It should be 
appreciated that other combinations of interference reduction light sources 
and optical filters can be used to reduce interference. For example, if a 
display is monochrome (e.g., green), supplemental light source 222 can be 
a red light source, and the optical filter can be a red color filter to allow the 
red light from supplemental light source 222 to pass and block the light 
from the monochrome display. 

In another embodiment, supplemental light source 222 may 
modulate in intensity at a different frequency than self-illuminated surface 
212. Image processor 306 is operable to electronically filter out unwanted 
illumination and process the illumination from supplemental light source 
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222. The illumination from supplemental light source 222 modulates at a 
known frequency, and image processor 306 is locked to the known 
frequency, ignoring the illumination from self-illuminated surface 212. 

5 Optical navigation componentry 165b of FIG. 2B also provides an 

electronic device for navigation and data input of a computer system without 
requiring a touch screen. Furthermore, embodiments in accordance with 
the invention are also useful where illumination from the display screen 
may cause interference with navigation and data input. Negating 

10 interference and providing additional illumination provide an increased 
range of applicability. Moreover, once the interference has subsided, the 
interference reduction light source can be turned off, providing the low 
power consumption mode as described at FIG. 2A. If the interference 
reduction light source is turned off, the optical filter may be moved or 

15 bypassed where it is an optical filter element (e.g., optical filter 230 of Figure 
2B or optical filter 308 of Figure 3) or turned off if is implemented 
electronically in image processor 306. 

20 Physical Structure of Exemplary Illuminated Surfaces upon which 

Embodiments in Accordance with the Invention may be Implemented 

FIG. 4A through FIG. 4C illustrate exemplary illuminated surfaces 
upon which embodiments in accordance with the invention may be 

25 implemented. FIG. 4A is a diagram of an exemplary cathode ray tube (CRT) 
400. CRT 400 comprises shadow mask 404, phosphor screen 410 and 
panel glass 412 within frame 408. Electron beam 402 travels through 
funnel glass 406 to illuminate shadow mask 404, phosphor screen 410 
and panel glass 412. FIG. 4B illustrates a blow up of an exemplary shadow 

30 mask 404. Shadow mask 404 typically has a micro textured structure 
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suitable for use as illuminated surface 206 of FIG. 2A. 

FIG. 4C is a diagram of an exemplary liquid crystal display (LCD) 420 
upon which an embodiment in accordance with the invention may be 
5 implemented. LCD 420 comprises polarizer layers 430, bottom glass layer 
432, transistor matrix 434, liquid crystal suspension 436, electrode layer 
438, and top glass layer 440. Backlight 450 provides illumination to LCD 
420. Transistor matrix 434 comprises a transistor at each pixel, and has a 
micro textured structure suitable for use as illuminated surface 206 of FIG. 
10 2A. Furthermore, LCD 420 may comprise diffuser or protection layer 442. 
Diffuser or protection layer 442 may be manufactured with sufficient micro 
textured structure suitable for use as self-illuminated surface 206 of FIG. 2A. 

While FIG. 4A through FIG. 4C illustrate specific examples upon 
15 which embodiments in accordance with the invention may be practiced, it 
should be appreciated that any layer or component of a display screen with 
sufficient micro textured structure as described above is suitable for use as 
self-illuminated surface 206 of FIG. 2A. In one embodiment in accordance 
with the invention, self-illuminated surface 206 is overlaid with a semi- 
20 transparent layer that has a detectable texture. For example, diffuser or 
protection layer 442 of FIG. 4C may be a semi-transparent layer that is 
placed over LCD 420. As such, any surface with sufficient illumination can 
be used as self-illuminated surface 206 of FIG. 2A. 

25 In one embodiment, the semi-transparent layer comprises unique 

positioning information providing absolute position information of the optical 
navigation device relative to the illuminated surface. The semi-transparent 
layer comprises a unique pattern such that an image taken by optical 
motion detection circuit 210 of FIG. 2A is operable to identify an absolute 

30 position on the illuminated surface. 
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Optical navigation componentry 165a of FIG. 2A provides an 
electronic device for navigation and data input of a computer system without 
requiring a touch screen. Furthermore, by eliminating the need for an 
5 internal light source, such as an LED, embodiments in accordance with the 
invention are useful where power consumption is an important concern, 
such as wireless devices with internal power sources. 

10 A Process of Optical Navigation on an Illuminated Surface Using an 

Electronic Device in Embodiments in Accordance with the Invention 

FIG. 5A and FIG. 5B illustrate a flow chart illustrating steps in a 
process 500 of optical navigation on an illuminated surface using an 

15 electronic device, in an embodiment in accordance with the invention. In 
one embodiment in accordance with the invention, process 500 is 
performed at optical navigation componentry of an electronic device (e.g., 
optical navigation componentry 165 of electronic device 130 of FIG. 1B) 
moving along an illuminated surface. Although specific blocks are 

20 disclosed in process 500, such blocks are exemplary. That is, the 

embodiments in accordance with the invention are well suited to performing 
various other blocks or variations of the blocks recited in FIG. 5. 

At block 502 of process 500, a start condition is initiated. In an 
25 embodiment in accordance with the invention, an image of an illuminated 
region is detected at an image detector (e.g., detector 304 of Figure 3). In 
one embodiment this is done with the aid of optical element 208 of FIG. 2A. 

At block 504, it is determined whether illumination from the 
30 illuminated surface is sufficient to with acquire a frame. In one embodiment 
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in accordance with the invention, a detector of an optical motion detection 
circuit (e.g., optical motion detection circuit 210 of FIG. 2B) is configured to 
determine whether illumination is sufficient to acquire a frame. Provided 
illumination from the illuminated surface is sufficient to acquire a frame, 
process 500 proceeds to block 514. 

Alternatively, provided illumination is insufficient to acquire a frame, 
as shown at block 506, additional illumination is provided onto the 
illuminated surface by a supplemental light source light source (e.g., 
supplemental light source 222 of FIG. 2B). 

At block 514, it is determined whether illumination from the 
illuminated surface is interfering with acquiring a frame. In one 
embodiment in accordance with the invention, a detector of an optical 
motion detection circuit (e.g., optical motion detection circuit 210 of FIG. 2B) 
is configured to determine whether illumination from the illuminated surface 
is interfering with acquiring a frame. Provided no interference is detected, 
process 500 proceeds to block 520. 

Alternatively, provided interference is detected, as shown at block 
516, interference reducing illumination is provided onto the illuminated 
surface by an interference reduction light source (e.g., supplemental light 
source 222 of FIG. 2B). At block 518, the illumination received from the 
illuminated surface, which now also comprises the interference reducing 
illumination, is filtered such that the optical motion detection circuit receives 
only the interference reducing illumination. It should be appreciated that the 
filtering may be performed by and optical filter element (e.g., optical filter 230 
of FIG. 2B or optical filter 308 of Figure 3) or may be performed electronically 
at the optical motion detection circuit. 
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At block 520, a reference frame from the illuminated surface is 
acquired. In one embodiment in accordance with the invention, a collection 
of digitized photo detector values is stored into an array of memory (not 
shown). As described above at FIG. 3, an optical motion detector circuit 
5 comprising a detector is operable to take an image of a portion of the 
illuminated surface, providing a reference frame. 

At block 522, a sample frame from the illuminated surface is 
acquired. This refers to the same action as block 520, except that the data 
10 is stored in a different array of memory, and may reflect motion relative of the 
electronic device to where it was when block 520 was performed. 



At block 524, correlation values are computed. In one embodiment in 
accordance with the invention, the nine (or perhaps twenty-five) correlation 
15 values are quickly computed by dedicated arithmetic hardware. At block 
526, a shift in the reference frame is predicted. The predicted shift can be 
taken as the amount of movement corresponding to the correlation at the 
preceding block 524. 

20 At block 528, a motion signal indicating the shift in position is output. 

In one embodiment in accordance with the invention, the motion signal 
comprises the change in location at the Y-axis (e.g., AX) and the change in 
location at the Y-axis (e.g., AY). The amount of motion since the last 
measurement cycle is noted here. The amount of shift needed to attain 

25 correlation is the desired amount. These values may be found by noticing 
which comparison frame actually correlated (assuming no interpolation). 
These "raw" AX and AY motion values may be accumulated into running 
values that are sent to the computer system at a lower rate than that at 
which the raw values of block 528 are produced. 

30 
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At block 530, it is determined whether a new reference frame is 
needed. Provided a new reference frame is not needed process 500 
proceeds to block 540. Alternatively, provided a new reference frame is 
needed, as shown at block 536, the present sample frame is stored as the 
5 reference frame. 

At block 540, the reference frame is shifted. The actual permanent 
shift of the values in the memory array representing the reference frame is 
performed. The shift is by the prediction amount, and data shifted away is 
10 lost. 

At block 542, it is determined whether the initialization conditions 
determined at blocks 504 and 514 need to be checked. In one 
embodiment, the initialization conditions are checked for every frame. In 

15 another embodiment, the initialization conditions are checked after a 

predetermined number of frames has been captured. For example, after 
ten frames have been captured, the initialization conditions are checked. It 
should be appreciated that the initialization conditions can be checked at 
any time, and are not limited to the described embodiments. Provided it is 

20 determined that the initialization conditions need to be checked, process 
500 returns to block 504. Alternatively, provided it is determined that the 
initialization conditions do not need to be checked, process 500 returns to 
block 520, where image capturing begins. 

25 While the invention has been described in particular embodiments, it 

should be appreciated that the invention should not be construed as limited 
by such embodiments, but rather construed according to the below claims. 
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